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IMAGE MATCHING SYSTEM AND IMAGE MATCHING METHOD AND 

PROGRAM 



5 BACKGROUND OF THE INVENTION 

1 . Field of the Invention 

The present invention relates to an image matching 
system for matching images based on image information of 
for example fingerprint images, still images, and moving 

10 images and an image matching method and program. 
2. Description of the Related Art 
Conventionally, as a method for matching images 
based on image information, various pattern matching 
techniques have been known. For example, the technique of 

15 detecting an amount of parallel movement and a rotation 
angle between a registered image and a matching image to 
be compared with and positioning the two based on the 
detection results has been proposed. Further, a pattern 
matching technique for performing a discrete Fourier 

20 transform for each of the registered image and the 

matching image, calculating a degree of correlation based 
on phase information of each of the obtained pattern data, 
and matching the images based on the correlation result 
has been known (for example, see Japanese Unexamined 

25 Patent Publication (Kokai) No. 1 10-55439) . 

Summarizing the problem to be solved by the 
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invention, the above pattern matching system detecting 
the amount of parallel movement and the rotation angle, 
however, cannot obtain correspondence of images and 
suitably match images when for example the scalings of 
5 the registered image and the matching image are different. 
Further, the pattern matching system performing the 
discrete Fourier transform and matching images based on 
the amplitude component of each of the obtained pattern 
data processes the registered image or the matching image 

10 to scale it based on the amplitude component and matches 
the images based on the processing results, but cannot 
detect the rotation angle. For this reason, an image 
matching system for matching two images even if there are 
differences in the amount of parallel movement, the 

15 rotation angle, and the scaling between images has been 
demanded. 

SUMMARY OF THE INVENTION 

An object of the present invention is to provide an 
image matching system and an image matching method and 
20 program able to match images even if there are 

differences in the amount of parallel movement, the 
rotation angle, and the scaling between images. 

To attain the above object, according to a first 
aspect of the present invention, there is provided an 
25 image matching system for matching a first image and a 
second image, comprising a correction information 



generating means for performing a Fourier transform and a 
log-polar coordinate transform to the first image and the 
second image and generating correction information of the 
first image based on the results of the Fourier transform 
5 and log-polar coordinate transform and a matching means 
for performing processing of correction of the first 
image based on the correction information generated by 
the correction information generating means, processing 
of correlation of the corrected first image and the 

10 second image, and processing of matching the results of 
the correlation processing. 

According to the first aspect of the present 
invention, the correction information generating means 
performs a Fourier transform and log-polar coordinate 

15 transform to the first image and the second image and 

generates correction information of the first image based 
on the results of the Fourier transform and the log-polar 
coordinate transform. The matching means performs 
processing of correction of the first image based on the 

20 correction information generated by the correction 

information generating means , processing of correlation 
of the corrected first image and second image, and 
processing of matching the results of the correlation 
processing. 

25 Preferably, the correction information generating 

means performs a further Fourier transform based on the 



results of the log-polar coordinate transform of the 
first image and the second image and generates scalar 
information and/or rotation information as the correction 
information based on correlation strength of the Fourier 
5 transformed first image and second image, and the 

matching means performs processing of correction of the 
first image based on the scalar information and/or the 
rotation information generated by the correction 
information generating means . 

10 More preferably, the correction information 

generating means generates the scalar information and/or 
rotation information as the correction information based 
on correlation strength of phase information the Fourier 
transformed first image and second image. 

15 Alternatively, the correction information 

generating means performs a Fourier-Mellih transform to 
the first image and the second image, performs processing 
for correlation between the Fourier-Mellin transformed 
first image and second image, and generates the scalar 

20 information and/or rotation information as the correction 
information. 

More preferably, the" matching means performs 
processing of correction of the first image based on the 
scalar information and/or the rotation information 

25 generated by the correction information generating means, 
performs processing for Fourier transform to the 



corrected first image and second image, and performs 
correlation processing based on the Fourier transformed 
first image and second image. 

Alternatively, more preferably, the matching means 
5 performs processing of correction of the first image 
based on the scalar information and/or the rotation 
information generated by the correction information 
generating means, performs processing for Fourier 
transform to the corrected first image and second image, 

10 and performs correlation processing based on phase 

information of the Fourier transformed first image and 
second image. 

Alternatively, the matching means generates 
parallel movement information of the corrected first 

15 image and second image based on a peak position of 
correlation strength of phase information of the 
corrected first image and second image, extracts common 
areas of the first image and' the second image based on 
the movement amount information, performs processing for 

20 correlation of the extracted common areas, and performs 
processing for matching the first image and the second 
image based on the results of the correlation processing. 

Alternatively, the matching means generates 
parallel movement information of the corrected first 

25 image and second image based on a peak position of 
correlation strength of phase information of the 



corrected first image and second image and performs 
processing for matching the first image and the second 
image when the parallel movement information is smaller 
than a predetermined amount of parallel movement. 
5 Further, to attain the above object, according to a 

second aspect of the present invention, there is provided 
an image matching method for matching a first image and a 
second image, comprising a first step of performing a 
Fourier transform and a log-polar coordinate transform to 

10 the first image and the second image and generating 

correction information of the first image based on the 
results of the Fourier transform and log-polar coordinate 
transform and a second step of performing processing of 
correction of the first image based on the correction 

15 information generated in the first step, processing of 
correlation of the corrected the first image and the 
second image, and processing of matching the results of 
the correlation processing. 

In the second aspect of the invention, there are 

20 the same preferable embodiments as explained in relation 
with the first aspect of the invention. In this case, 
read "correction information generating means" as "first 
step" and "matching means" as "second step". 

Further, to attain the above object, according to a 

25 third aspect of the present invention, there is provided 
a program to be executed by an information processing 



apparatus for performing processing for matching a first 
image and a second image, comprising a first routine for 
performing a Fourier transform and a log-polar coordinate 
transform to the first image and the second image and 
5 generating correction information of the first image 
based on the results of the Fourier transform and log- 
polar coordinate transform and a second routine for 
performing processing for correction the first image to 
the correction information generated by the first routine, 

10 processing for correlation of the corrected the first 

image and the second image, and processing for matching 
the results of the correlation processing. 

In the third aspect of the invention, there are the 
same preferable embodiments as Explained in relation with 

15 the first aspect of the invention. In this case, read 
"correction information generating means" as "first 
routine" and "matching means" as "second routine". 
BRIEF DESCRIPTION 1 OF THE DRAWINGS 

These and other objects and features of the present 
20 invention will become clearer from the following 

description of the preferred embodiments given with 
reference to the attached drawings, wherein: 

FIG. 1 is a functional block diagram of an image 
matching system according to a first embodiment of the 
25 present invention; 

FIG. 2 is a software-like functional block diagram 



of the image matching system shown in FIG. 1; 

FIG. 3 is a view for explaining an operation of a 
Fourier-Mellin transform unit shown in FIG. 2; 

FIG. 4 is a view for explaining correlation value 
5 generation processing of a correlation value generation 
unit; 

FIGS. 5A to 5F are views for explaining a 
difference between a self correlation method and a phase 
only correlation method; 
10 FIGS. 6A to 6C are views for explaining a 

correlation strength distribution where there is 
deviation in parallel movement between two images in the 
phase only correlation method; 

FIGS. 7A to 7C are views for explaining a 
15 correlation strength distribution where there is 

deviation in rotation between two images in the phase 
only correlation method; 

FIG. 8 is a flow chart for explaining the operation 
of the image matching system shown in FIG. 1; 
20 FIGS. 9A to 9F are views for explaining a general 

polar coordinate transform in the case of fingerprint 
images as the regis tereld image and the matching image; 

FIGS. 10A to IGF are views for explaining a 
Fourier-Mellin transform in the case of fingerprint 
25 images as the registered image and the matching image; 

FIG. 11 is a functional block diagram of an image 



matching system according to a second embodiment of the 
present invention; 

FIG. 12 is a flow chart for explaining the 
operation of the image matching system according to the 
5 second embodiment shown in FIG. 11; 

FIG. 13 is a functional block diagram of an image 
matching system according to a third embodiment of the 
present invention; 

FIG. 14 is a view for explaining the operation of 
10 detecting an amount of parallel movement by a parallel 
movement detection unit; 

FIG. 15 is a flow chart for explaining the 
operation of the image matching system shown in FIG. 14; 

FIG. 16 is a view of part of the functional blocks 
15 of an image matching system according to a fourth 
embodiment of the present invention; and 

FIG. 17 is a flow chart for explaining the 
operation of the image matching system according to the 
fourth embodiment. 
20 DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Preferred embodiments of the present invention will 
be described in detail below while referring to the 
attached figures. FIG. I is a functional block diagram of 
an image matching system according to a first embodiment 
25 of the present invention. The image matching system 1 

according to the present embodiment has, for example as 



shown in FIG. 1, an image input unit 1, a memory 12, an 
FFT processing unit 13, a coordinate transform unit 14, a 
CPU 15, and an operation processing unit 16. The image 
input unit 11, the memory 12, the FFT processing unit 13, 
5 the coordinate transform unit 14, and the CPU 15 are 

connected by a bus BUS. The operation processing unit 16 
is connected to the CPU 15. 

The image input unit 11 is an input unit for 
inputting an image from the outside. For example, the 

10 image input unit 11 receives as input a registered image 
AIM and an image to be compared with the registered image 
AIM (referred to as a "matching image RIM") are input. 
The memory 12 stores the images etc. input from the image 
input unit 11. For example, the memory 12 stores the 

15 registered image AIM, the matching image RIM, and a 

program P.* The program P includes routines to be executed 
by for example the CPU 15 for the correlation processing, 
the correction processing, the matching processing, etc. 
according to the present invention. The FFT processing 

20 unit 13 performs two-dimensional Fourier transform 

processing based on the images stored in the memory 12 
under the control of for example the CPU 15 and outputs 
the processing results to the coordinate transform unit 
14 : and the CPU 15. 

25 The coordinate transform unit 14 transforms the 

results to log-polar coordinates based on the results of 
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the two-dimensional Fourier transform processing from the 
FFT processing unit under the control of for example the 
CPU 15 and outputs the processing results to the CPU 15. 
The operation processing unit 16 performs the 
5 predetermined processing for releasing an electronic lock 
etc. when for example the two images coincide based on 
the results of the matching processing of the CPU 15 
mentioned later. The CPU 15 performs the matching 
processing according to the embodiment of the present 

10 invention based on the program P , the registered image 

AIM, and the matching image RIM stored in for example the 
memory 12. Further, the CPU 15 controls the image input 
unit 11, the memory 12, the FFT processing unit 13, the 
coordinate transform unit 14, the operation processing 

15 unit 16, etc. to execute the processing according to the 
present embodiment. 

FIG. 2 is a software-like functional block diagram 
of the image matching system shown in FIG. 1. The CPU 15 
controls the FFT processing unit 13, the coordinate 

20 transform unit 14, etc. and performs the processing based 
on for example the functional blocks as shown in FIG. 2. 
The CPU 15 has, as shown in FIG. 2, a scalar information- 
rotation information unit 21, a correction unit 22, a 
parallel movement unit 23, and' a matching unit 24. 

25 The scalar information-rotation information unit 21 

generates scalar information and/or rotation information 
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based on the registered image AIM and the matching image 
RIM and outputs the same to the correction unit 22 . The 
scalar information includes information indicating the 
scaling of the registered image AIM and the matching 
5 image RIM. The rotation information includes information 
indicating the rotation angle of the registered image AIM 
and the matching image RIM. 

In more detail, for example the scalar information- 
rotation information unit 21 has a Fourier -Mel 1 in 

10 transform unit 211, a phase only correlation unit 212, 

and a scalar information-rotation information generation 
unit 213. The Fourier-Mellin transform unit 211 performs 
the Fourier-Mellin transform mentioned later based on 
each image information arid outputs signals SA211 and 

15 SR211 indicating the transform results to the phase only 
correlation unit 212. 

In more detail, the Fourier-Mellin transform unit 
211 has Fourier transform units 21111 and 21112, log 
transform units 21121 and 21122, and log-polar coordinate 

20 transform units 21131 and 21132. 

The Fourier transform unit 21111 performs the 
Fourier transform as shown in Equation (1) , where the 
registered image AIM is £1 (m, ri) in the case where for 
example the registered image AIM is an image comprised of 

25 N x N images , generates Fourier image data Fl(m,n), and 
outputs the same to the log transform unit 21121. The 



Fourier transform unit 21112 performs the Fourier 
transform as shown in Equation (2) where the matching 
image RIM is f2(m,n) in the case where for example the 
matching image RIM is an image comprised by N x N images, 
5 generates Fourier image data F2(m,n) / and outputs the 
same to the log transform unit 21122. 

The Fourier image data Fl(m,n) is comprised by an 
amplitude spectrum A(u,v) and a phase spectrum 0(u,v) as 
shown in Equation (1) , while the Fourier image data 
10 F2(m / n) is comprised by an amplitude spectrum B(u,v) and 
a phase spectrum <p(u,v) as shown in Equation (2) . 

Fl(u, v) = § Z M™' n)e- J2 *« mu+mf)/ ' N) = A(u, v)e j&iuy) ( 1 ) 

m=0 «=0 

F2(u, v) = ^ Z /2(w, ^-^a-^w) = B(JJ v)ff y*(».v) ( 2 , 

m =0 n=0 

15 

The log transform units 21121 and 21122 perform log 
processings based on the amplitude components of the 
Fourier image data Fl(m,n) and F2(m,n) generated at the 
Fourier transform units 21111 and 21112. The log 

20 processings of the amplitude components emphasize high 
frequency components including detailed characteristic 
information of the image data. 

In more detail, the log transform unit 21121 
performs log processing based on the amplitude component 

25 A(u,v) as shown in Equation (3) to generate A' (u,v) and 
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outputs the same to the log-polar coordinate transform 
unit 21131. The log transform unit 21122 performs log 
processing based on the amplitude component B(u,v) as 
shown in Equation (4) and outputs the same to the log- 
5 polar coordinate transform unit 21132. 

A' <u,v)=log(|A(u,v> |+1) (3) 
B' <u,v)=log(|B(u,v) |+1) (4) 
The log-polar coordinate transform units 21131 and 
21132 transform the signals output from the log transform 
10 units 21121 and 21122 to a log-polar coordinate system 
(for example log-r, 0) . In general, for example, when 
defining a point (x,y) as shown in Equations (5) and (6) , 
p=log(r) in the case of r=e p , and there is an unambiguous 
(log(r) ,0) corresponding to' any point (x, y) . Due to this 
15 nature, the log-polar coordinate transform units 21131 
and 21132 perform coordinate transforms. 

x=e p cos© (5) 
y=e v sin©(O<0<2n) (6) 
In more detail, the log-polar coordinate transform 
20 units 21131 and 21132 define a set (r^Gj) shown in 

Equation (7) and a function f (r± , 9j) shown in Equation 
(8). 

(ri,0 3 > : e 3 =2nj/N-n <7) 

f (r ± , Gj) = (riCOs9j+N/2 , riSinGj+N/2) 
25 (i=0, 1,..., N-l, j=0> 1,..., N-l) (8) 

The log-polar coordinate transform units 21131 and 



21132 perform the log-polar coordinate transforms as 
shown in Equations (9) and (10) on the image data A' (u,v) 
and B' (u,v) by using the set (r^Gj) and the function 
f(r±,e 3 ) defined by Equations (7) and (8), generate 
5 pA(ri,9 3 ) and pB(r i/ 0 j ) , and output them as the signal 

SA211 and the signal SR211 to the phase only correlation 
unit 212. 

pMr ±/ e a )=A' (f (r^e^)) (9) 
pB<ri,6j)=B' (f (ri,^)) (10) 

10 FIG. 3 is a view for explaining the operation of 

the Fourier-Mellin transform unit 211 shown in FIG. 2. 
The image fl(m,n) and the image f2(m,n) include 
rectangular areas Wl and W2 having different 
predetermined angles with respfect to for example the x,y 

15 axes. The Fourier-Mellin transform unit 211, for example, 
as shown in FIG. 3, performs a Fourier transform on the 
image fl(m,n) by the Fourier transform unit 21111 to 
generate the Fourier image data A' (u, v) and generates 
the image data pA(r,9) by the log transform unit 21121 

20 and the log-polar coordinate transform unit 21131. In the 
same way, it performs a Fourier transform on the image 
f2(m,n) by the Fourier transform unit 21112 to generate 
the Fourier image data B' (u,v) and generates the image 
data pB(r,9) by the log transform unit 21122 and the log- 

25 polar coordinate transform unit 21132. 

As explained above, the images fl(m,n) and f2(m,n) 



are transformed from cartesian coordinates to a log-polar 
coordinate system (also referred to as "Fourier-Mellin 
space") by the Fourier transform and the log-polar 
coordinate transform. In the Fourier-Mellin space, a 
5 component moves along the log-r axis in accordance with 
the scaling of the image and moves along the 8 axis in 
accordance with the rotation angle of the image. By using 
this nature, the scaling (scalar information) and the 
rotation angle of the image fl(m,n) and f2(m,n) can be 

10 found based on the movement amount along the log-r axis 
and the movement amount along the axis 9 in the Fourier- 
Mellin space. 

The phase only correlation unit 212 finds the 
amounts of parallel movement by the phase only 

15 correlation method using for example a phase only filter 
(SPOMF: Symmetric phase-only matched filter) based on the 
signals SA2 12 and SR212 indicating the pattern data 
output from the Fourier-Mellin transform unit 211. The 
phase only correlation unit 212 has, for example as shown 

20 in FIG. 1, Fourier transform units 2120 and 2121, a 

combining unit 2122, a phase extraction unit 2123, and an 
inverse Fourier transform unit 2124. 

The Fourier transform units 2120 and 2121 perform 
the Fourier transforms by Equations (11) and (12) based 

25 on the signals SA211 (pA(m,n)) and SR211 (pB(m,n)) output 
from the log-polar coordinate transform units 21131 and 



21132. Here, X(u,v) and Y(u,v) are Fourier coefficients. 
The Fourier coefficient X(u,v) is comprised of an 
amplitude spectrum C(u,v) and a phase spectrum 0(u,v) as 
shown in Equation (11) . The Fourier coefficient Y(u,v) is 
5 comprised of an amplitude spectrum D(u,v) and a phase 
spectrum <p(u,v) as shown in Equation (12). 

X(u, v) = £ £ pA(m, w ) e -> 2 *«™ )'"> = C (u, v)e mu ' v) (11) 

m=0 n=0 

F2(u, v) = § Z /2(w, W )e->2*a~ >'"> = v )e" ( "- v) ( 12 ) 

The combining unit 2122 combines the X(u,v) and 

10 Y(u,v) generated at the Fourier transform units 2120 and 
2121 and finds the correlation. For example, the 
combining unit 2122 generates X(u, v) Y* (u,v) and outputs 
the same to the phase extraction unit 2123. Here, Y*(u,v) 
is a complex conjugate of Y(u,v) . 

15 The phase extraction unit 2123 removes the 

amplitude component based on the combined signal output 
from the combining unit 2122 to extract the phase 
information. For example, the phase extraction unit 2123 
extracts the phase component Z (u,v) =e 3(e<u ' v) " q>(u ' v>) based on 

20 for example X(u,v) , Y* (u,v) . 

The extraction of the phase information is not 
limited to the above format. For example, it is also 
possible to extract the phase information based on the 
outputs of for example the Fourier transform units 2120 

25 and 2121 and Equations (13) and (14), then combine only 
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the phase components to generate Z(u,v) as shown in 
Equation (15) . 

X' <u,v)= e 36(u ' v> (13) 
Y' (u,v)=e 3<p<u ' v) (14) 
5 Z(u,v)=X' (u,v) (Y' (u,v) )=e j(e(u v) " <p(u v)) (15) 

The inverse Fourier transform unit 2124 performs 
the inverse Fourier transform processing based on the 
signal Z(u,v) of only the phase information output from 
the phase extraction unit 2123 to generate a correlation 
10 strength image. In more detail, the inverse Fourier 

transform unit 2124 performs inverse Fourier transform 
processing based on the signal Z(u,v) shown in Equation 
(16) to generate a correlation strength image G(p,q) . 

N-l N-l N-l N-l 

G(P <?) = 2 ^ (Z(u v))e' 2 ' r(( " /H " v * )/Ar) = 2 2 (e y( ^ (w ' v) "^ ( " ,v)) )e >2/r(( " p+V9)/Ar) 

u=0 v=0 u=0 v=0 

15 (16) 
The scalar information-rotation information 
generation unit 213 detects an amount of deviation of the 
peak position in the correlation strength image G(p,q) 
generated by the inverse Fourier transform unit 2124 from 

20 the image center since the amount of deviation is 

equivalent to the amount of parallel movement between 
pattern data obtained as a result of performing Fourier- 
Mellin transforms on the registered image AIM and the 
matching image RIM and thereby generates the correction 

25 information S21 including the scalar information 



(scaling) and the rotation angle information of" the 
matching image RIM with respect to the registered image 
AIM, 

The correction unit 22 performs correction of the 
5 matching image RIM based on the correction information 
S21 output from the scalar information-rotation 
information generation unit 213 of the scalar 
information-rotation information unit 21. In more detail, 
the correction unit 22 performs scaling processing and 

10 rotation processing on the matching image RIM based on 

the scalar information and the rotation angle information 
included in the correction information S21 and outputs 
the result to the parallel movement unit 23. By the 
correction processing of the correction unit 22, the 

15 differences in the scaling and rotation components 

between the registered image RIM and the matching image 
RIM are eliminated. For this reason, only the parallel 
movement component remains as the difference between the 
registered image AIM and the corrected matching image RIM. 

20 The parallel movement unit 23 detects the parallel 

movement component 1 between the registered image AIM and 
the corrected matching image RIM mentioned above and the 
correlation value thereof. This detection is by the phase 
only correlation method using for example the phase only 

25 filter mentioned above. In more detail, the parallel 
movement unit 23 has Fourier transform units 2311 and 



2312, a combining unit 232, a phase extraction unit 233, 
and an inverse Fourier transform unit 234 . 

The Fourier transform units 2311 and 2312, the 
combining unit 232, the phase extraction unit 233, and 
5 the inverse Fourier transform unit 234 have the same 
functions as those of the Fourier transform units 2120 
and 2121, the combining unit 2122, the phase extraction 
unit 2123, and the inverse Fourier transform unit 2124 of 
the phase only correlation unit 212, so will be only 

10 simply explained. 

The Fourier transform unit 2311 performs a Fourier 
transform on the registered image AIM and outputs the 
result to the combining unit 232. At this time, it is 
also possible to store the registered image AIM Fourier 

15 transformed at the Fourier transform unit 21111 in 
advance in the memory 12 and output the same to the 
combining unit 232. By doing this, the processing is 
reduced since double Fourier transform processing is not 
performed. The Fourier transform unit 2312 performs a 

20 Fourier transform on the corrected image S22 from the 

correction unit 22 and outputs the Fourier image as the 
processing result to the combining unit 232. 

The combining unit 232 combines the Fourier images 
S2311 and S2312 output from the Fourier transform units 

25 2311 and 2312 and outputs the combined image S232 to the 
phase extraction unit 233. The phase extraction unit 233 



extracts the phase information as mentioned above based 
on the combined image S232 and outputs a signal S233 to 
the inverse Fourier transform unit 234. The inverse 
Fourier transform unit 234 performs an inverse Fourier 
5 transform based on the signal S233, generates a 

correlation strength image (correlation image data) , and 
outputs it as a signal S23 to the matching unit 24. 

The matching unit 24 matches the registered image 
AIM and the matching image RIM based on the signal S23 

10 output from the parallel movement unit 23. For example, 
the matching unit 24 has a correlation value generation 
unit 241 and a judgment unit 242. The correlation value 
generation unit 241 generates a correlation value based 
on : a correlation strength distribution in the signal S23 

15 output from the parallel movement unit 23. 

FIG. 4 is a view for explaining the correlation 
value generation processing of the correlation value 
generation unit 241. The correlation value image data of 
the signal S23 output from the parallel movement unit 23 

20 has a sharp correlation peak PP at the center, for 
example, as shown in FIG. 4. The height of the 
correlation peak indicates the degree of similarity 
between the registered image AIM and the matching image 
RIM before the Fourier transform. For this reason, the 

25 correlation peak PP is used as the judgment reference of 
the matching of the images . 
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In more detail , the correlation value generation 
unit 241 defines for example a peripheral portion of the 
peak position as the correlation area R and generates a 
correlation value S241 based on a sum of the correlation 
5 values in the correlation area R and the sum of the 

correlation values in the entire area AR. For example, 
the correlation area R is an area of 5 x 5 pixels of the 
peripheral portion of the peak position PP as shown in 
FIG. 4. 

10 The judgment unit 242 judges the matching of the 

images based on the correlation value S241 generated by 
the correlation value generation unit 241 and outputs 
data S242 indicating the result of the judgment. For 
example, the 1 judgment unit 242 compares it with a 

15 threshold value determined in advance. When the 

correlation value S241 is larger than the threshold value, 
it judges that the registered image AIM and the matching 
image RIM coincide, while when it is smaller than the 
threshold value, it judges they do not coincide. 

20 A detailed explanation of the processing for 

detection of the amount of parallel movement in the 
phase only correlation method will be given next. For 
example, an original image fl <m, n) , an original image 
f2(m,n)> and an image f 3 (m,n)=f 2 (m+a,n+P) obtained by 

25 parallel movement of the image f2(m,n) are Fourier 

transformed to generate Fourier coefficients Fl(u,v), 
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F2(u,v), and F3(u,v) as shown in Equations (17) to (19) 

N-l N-\ 



Fl(u, v) = X E /Kw, n)e- J2 * (( " )/N) = A(u, v)e m "- v) (17) 

m=0 n=0 

F2(u, v) = ^ H f2(m, n ) e - j2 *« mu+m)/ ^ = B(u, v)e m " v) (18) 

F3(«, v) = f H /2(m + a,n + fo-W-*"*'") = v ) e ><•(«. ^ 2l (»*« w ) 

nj=0 n-0 

5 (19) 
Based on the Fourier coefficients Fl(u,v) to 
F3(u,v), as shown in Equations (20) to (22), the phase 
images F' l(u,v) to F'3(u,v) of only the phase information 
are generated, 
10 F'l(u,v)=e 3e(u ' v) (20) 

F'2(u,v)=e j<p(u ' v> (21) 
F' 3 ( U/V )=e :i((p(u ' v)+2n(au+pv>/N) (22) 
A phase Z12(u,v) of the phase image of the 
correlation between the phase image F'l(u,v) and the 
15 phase image F'2(u,v) is calculated as shown in Equation 

(23) , while a correlation Z13(u,v) of the phase image of 
the correlation between the phase image F'l(u,v) and the 
phase image F'3(u,v) is calculated as shown in Equation 

(24) : 

20 Z12 (u,v)=F' 1 (u,v) (F'2 (u,v) ) *=e 3<e(u ' v) ~ <p<u ' v)) (23) 

Z13 (u,v)=F' 1 (u,v) (F' 3 (u,v) ) *= e 3(0(u ' v) - <p(u ' v) - 2n(au+pv>/N) 

(24) 

A correlation strength image G12(r,s) of the 
correlation Z12 and a correlation strength image G13(r,s) 
25 of the correlation Z13 are ' calculated as shown in 
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Equations (25) and (26) : 

Gl2(r,5) = ^ J] (Z12(M 3 V))e y2,r(( " r+V,)W) =2 ^ ( e y(«(».vW(«.v»^/2 > r«^v f )/JV) 
tt =0 v=0 u=0 v=0 

(25) 



Gl3(r,*) = £ Z <,Z\3(u,v))e i2a « ur+vsVN ^ 



u=0 v=0 

g ^ ^>(^(«,v)-^(tt,v)-2^(a«+^/^)^>2^((«r+v5)/N) (26) 

M =0 v=0 

As shown in Equations (25) and (26) , when the image 
f3(m,n) is deviated by exactly (+ot, + p) in comparison with 
the image f2(m,n), with the phase only correlation method, 
the peak of the correlation strength is generated at the 

10 position deviated by exactly (-ct,-|3) in the correlation 
strength image. From the position of this correlation 
strength position, the amount of parallel movement 
between two images can be found. Further, by using this 
phase only correlation method in the Fourier-Mellin space, 

15 the amount of parallel movement in the Fourier-Mellin 

space can be detected. This amount of parallel movement 
corresponds to the scalar information and the rotation 
angle information in real space as mentioned above. 
FIGS. 5A to 5F are views for explaining a 

20 difference between the iself correlation method and the 
phase only correlation method. In the self correlation 
method, as shown in for example FIGS. 5A and 5B, when the 
image IM1 and the image IM2 the same as the image IM1 are 
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subjected to Fourier transforms to generate the self 
correlation function SGI, as shown in FIG. 5C, a 
correlation strength distribution having a peak of high 
correlation strength and a small correlation strength in 
the peripheral portion thereof is obtained. In FIG. 5C, 
the ordinate indicates the correlation strength. 

On the other hand, in the phase only correlation 
method, as shown in FIGS. 5D and 5E, when the image IMi 
and the image IM2 the same as the image IMI are subjected 
to Fourier transforms to finding correlation for only the 
phase information, as shown in FIG. 5F, a correlation 
strength distribution having only a sharp peak having a 
high correlation strength is obtained. In this way, clear 
information concerning the correlation in comparison with 
the self correlation method can be obtained in by phase 
only correlation method. In FIG. 5F, the ordinate (z 
axis) indicates the correlation strength, and an x axis 
and a y axis indicate the amount of deviation. 

FIGS. 6A to 6C are views for explaining the 
correlation strength distribution of a case where there 
is deviation in parallel movement between two images in 
the phase only correlation method. For example, the 
correlation strength distribution in the phase only 
correlation method of the image IMI and the image IM3 
obtained by parallel movement by several pixels from the 
image IMI as shown in FIGS. 6A and 6B is formed so that 



the sharp peak which has a high correlation strength is 
located at a position deviated from the peak position in 
the correlation image data shown in FIG. 5F by exactly 
the amount of parallel movement as shown in for example 
5 FIG. 6C. However, the peak strength shown in FIG. 6C is 

smaller in comparison with the peak strength shown in FIG. 
5F. This is because the pixel area where the images IM1 
and IM3 coincide is smaller than the images IM1 and IM2 . 
FIGS. 7A to 7C are views for explaining a 

10 correlation strength distribution when there is rotation 
deviation between two images in the phase only 
correlation method. In the correlation strength 
distribution in the phase only correlation method of the 
image IM1 and the image 4 rotated by several degrees from 

15 the image IM1 as shown in FIGS. 7A and 7B, for example, 
as shown in FIG. 7C, a correlation strength distribution 
having a weak correlation strength is obtained. When the 
phase only correlation method is simply used, due to the 
rotation deviation, it is difficult to detect correlation. 

20 For this reason, the image matching system 1 

according to the present embodiment performs Fourier- 
Mel 1 in transforms on the registered image AIM and the 
matching image RIM and usss the phase only correlation 
method in the Fourier-Mellin space so as to detect the 

25 amount of parallel movement and detect the scalar 

information and the rotation angle information of the 
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registered image AIM and the matching image RIM. Based on 
the information, the scaling and the rotation angle of 
the matching image are corrected. The deviation of 
parallel movement between the corrected matching image 
RIM and registered image AIM is detected by the phase 
only correlation method. The matching of images is 
simultaneously carried out based on the correlation value 
peak . 

FIG. 8 is a flow chart for explaining the operation 
of the image matching system 1 shown in FIG. 1. The 
operation of the image matching system 1 having a 
configuration explained above will be simply explained 
with reference to FIG. 8. For example, the image input 
unit 11 inputs the registered image AIM and the matching 
image RIM and stores the image data in the memory (ST1) . 
Here, in order to find the scalar information (scaling) 
and the rotation angle information of the matching image 
RIM with respect to the registered image AIM, the 
registered image AIM is read from the memory 12 (ST2) . 
The Fourier transform unit 21111 of the scalar 
information-rotation information unit 21 performs the 
Fourier transform processing (ST3) , then the Fourier 
image data S21111 is stored in the msmory 12 (5T4) . The 
amplitude component' in the Fourier image data S21111 is 
subjected to log processing by the log transform unit 
21121, then the log-polar coordinate transform unit 21131 



transforms this to the log-polar coordinate system (ST5) . 

The matching image RIM is read out from the memory 
12 (ST6) and processed by Fourier transform by the 
Fourier transform unit 21112 in the same way as above 
5 (ST7) . The amplitude component in the Fourier image data 
S21112 is subjected to log processing by the log 
transform unit 21112 and is transformed to the log-polar 
coordinate system by the log-polar coordinate transform 
unit 21132 (ST8) . 

10 The image signals (also referred to as the pattern 

data) SA211 and SR211 obtained as the result of the 
Fourier-Mellin transforms on the registered image AIM and 
the matching image RIM are Fourier transformed by the 
Fourier transform units 2120 and 2121 of the phase only 

15 correlation unit 212 (ST9) , and combined by the combining 
unit 2122 (ST10) . The amplitude component is removed from 
the combined signal by the phase extraction unit 2123 
(ST11) . The remaining phase component is subjected to 
inverse Fourier transform processing by the inverse 

20 Fourier transform unit 2124 (ST12) . ' Based on the amount 
of deviation of the peak position oif the obtained 
correlation image data from the image center, correction 
information including scalar information and rotation 
information is generated by the scalar information— 

25 rotation information generation unit 213 (ST13) . 

The correction unit 22 performs processing for 
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correction of scaling and rotation of the matching image 
based on the correction information and eliminates the 
scaling component and the rotation component between the 
images (ST14) . The remaining difference is only the 
5 parallel movement component and is detected by using the 
phase only correlation method. 

The corrected matching image RIM is Fourier 
transformed by the Fourier transform unit 2312 of the 
parallel movement unit 23 (ST15) to generate the Fourier 

10 image data S2312, the Fourier transformed registered 

image AIM stored in the memory 12 is read out (ST16) , and 
combined data S232 is generated by the combining unit 232 
(ST17) . At this time, it is also possible to perform a 
Fourier transform on the registered image AIM by the 

15 Fourier transform unit 2311 to generate the Fourier image 
data 2311 and input it to the combining unit 232. The 
amplitude information in the combined data S232 is 
removed by the phase extraction unit 233 (ST18) . The 
remaining phase information is input to the inverse 

20 Fourier transform unit 234 (ST19) . 

Based on the correlation image data S23 generated 
by the above processings, a value obtained by dividing 
the sum of the correlation values of for example 5x5 
pixels in the vicinity of for example the peak position 

25 by the sum of all correlation values from the correlation 
value generation unit 241 is generated as the correlation 



value S241 (ST20) . The judgment unit 242 judges whether 
or not the correlation value S241 is larger than a 
predetermined threshold value (ST21) . When it is larger 
than the threshold value, it judges that the registered 
5 image AIM and the matching image RIM coincide (ST22) , and 
predetermined processing is carried out by the operation 
processing unit 16. For example, when the image matching 
system 1 according to the present embodiment is used for 
a fingerprint matching system in the security field, 
10 processing for releasing an electronic lock is carried 
out. 

On the other hand, at step ST21, when the judgment 
unit 242 judges that the correlation value S241 is 
smaller than the threshold value (ST23) , it recognizes 

15 that the registered image AIM and the matching image RIM 
do not coincide and terminates the series of processing. 

As explained above, in the present embodiment, 
since provision was made of the Fourier-Mellin transform 
unit 211 for performing Fourier transforms and log-polar 

20 coordinate Fourier transforms on the registered image AIM 
and the matching image RIM, the phase only correlation 
unit 212 and the scalar information-rotation information 
generation unit 213 for generating correction information 
including scalar information and rotation information 

25 based on the correlation strength of the phase 

information based on the transform result, the correction 



unit 22 for correcting the matching image RIM in 
accordance with the image information based on the 
correction information, the parallel movement unit 23 for 
performing phase only correlation processing based on the 
5 image of the result of the correction processing and the 
registered image, the correlation value generation unit 
241 for generating a correlation value based on the 
correlation image data generated by the phase only 
correlation processing, and the judgment unit 242 for 

10 performing processing for judgment concerning matching 
based on the correlation value, even if there are 
differences in the amount of parallel movement, the 
rotation angle, and the scaling between the images, 
matching can be carried out. 

15 That is, by performing the log transform processing 

on the Fourier image, performing a log-polar coordinate 
transform, and performing the phase only correlation 
processing in the Fourier-Mellin coordinate system, even 
if ' there are differences in the amount of parallel 

20 movement, the rotation angle, and the scaling between the 
images, matching can be carried out. 

FIGS . 9A to 9F are views' for explaining a general 
polar coordinate transform when fingerprint images are 
utilized as the registered image and the matching image. 

25 For example, the fingerprint registered image AIM shown 
in FIG. 9A and the fingerpririt matching image RIM shown 
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in FIG. 9B smaller than the fingerprint registered image 
AIM and having deviation in rotation angle are subjected 
to Fourier transform processing to generate Fourier 
images IM11 and IM12 shown in FIGS, 9C and 9D. When the 

5 polar coordinate transform is simply carried out without 

* 

performing a log transform, images IM111 and IM121 are 
generated for example as shown in FIGS. 9E and 9F. In the 
images IM111 and IM121, however, the radius vector 
direction r cannot be judged. 

10 FIGS. 10A to 10F are views for explaining the 

Fourier-Mellin transform in the case of fingerprint 
images as the registered image and the matching image. 
For example, the fingerprint registered image AIM shown 
in FIG. 10A and the fingerprint matching image RIM shown 

15 in FIG. 10B smaller than the fingerprint registered image 
AIM and having deviation in rotation angle are subjected 
to Fourier transform processing to generate the Fourier 
images IM11 and IM12 shown in FIGS. 10C and 10D. When log 
transforms aire carried out and log-polar coordinate 

20 transforms are carried out, images IM112 and IM122 are 
generated for example as shown in FIGS. 10E and 10F. In 
the images IM112 and IM1:22 , high frequency components are 
emphasized in log-r direction, therefore correlation 
between images can be easily found. 

25 FIG. 11 is a functional block diagram of an image 

matching system according to a second embodiment of the 
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present invention. An image matching system la according 
to the present embodiment detects positional deviation of 
the registered image AIM and the matching image RIM, 
corrects the positional deviation based on the detection 
5 result, then extracts the common partial areas of the two 
images, finds the correlation of each of the extracted 
common partial areas, and performs matching based on the 
correlation results . 

The image matching system la has the same 

10 configuration as the image matching system 1 according to 
the first embodiment in terms of hardware. The software- 
like difference between the image matching system la and 
the image matching system 1 according to the! first 
embodiment resides in that, as shown in FIG. 11, a 

15 parallel movement detection unit 251, a common partial 
area extraction unit 252, and a phase only correlation 
unit 253 are provided. The rest of the components are 
given the same reference numerals and explanations 
thereof are omitted. 

20 The parallel movement detection unit 251 detects 

the amount of parallel movement based on the signal S23 
indicating the correlation strength image output from the 
parallel movement unit 23 in FIG. 11 and outputs the same 
to the partial extraction unit 2522. The amount of 

25 parallel movement is obtained by detecting the amount of 
deviation of the peak position from the image center in 
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the correlation strength image. This amount of deviation 
is equivalent to the amount of deviation concerning 
parallel movement between the registered image AIM and 
the matching image RIM. 

The common partial area extraction unit 252 has a 
partial extraction unit 2521 and a partial extraction 
unit 2522. The partial extraction unit 2521 extracts a 
partial pixel area set in advance from the interior of 
the registered image AIM and outputs the same as the 
signal S2521 to the Fourier transform unit 2531 as shown 
in FIG. 11. 

The partial extraction unit 2522 extracts a common 
partial pixel area corresponding to the partial pixel 
area of the above registered image AIM from the interior 
of the matching image RIM based on the signal S251 output 
from the parallel movement detection unit 251 and outputs 
the same as the signal S2522 to the Fourier transform 
unit 2532. 

The phase only correlation unit 253 finds 
correlation by the phase only correlation method, 
generates the correlation image data S253 in the common 
partial area, and outputs it to the matching unit 24. In 
more detail, for example the phase only correlation unit 
253 has Fourier transform units 2531 and 2532, a 
combining unit 2534, a phaise extraction unit 2535, and an 
inverse Fourier transform unit 2536. The components have 



functions the same as those of the phase only correlation 
unit 212 mentioned above, so explanations will be omitted. 

FIG. 12 is a flow chart for explaining the 
operation of the image matching system according to the 
5 second embodiment shown in FIG. 11. Only the difference 
of the operation of the image matching system la will be 
explained with reference to FIG. 12. The processing of 
steps ST1 to ST19 is the same as in the operation of the 
image matching system 1 according to the first embodiment. 
10 At step ST120, an amount of parallel movement S251 is 

detected from the signal S23 indicating the correlation 
strength image generated at the inverse Fourier transform 
unit 234 at step ST19 by the parallel movement detection 
unit 251. 

15 At step ST121, the partial extraction unit 2521 

extracts the partial area set in advance from the 
registered image AIM. This partial area is Fourier 
transformed by the Fourier transform unit 2531 (ST122) . 
The partial extraction unit 2522 extracts the common 

20 partial area corresponding to the partial area of the 
above registered image AIM from the matching image RIM 
based on the amount of parallel movement S251 (ST123) . 
Fourier transform processing is then carried out by the 
Fourier transform unit 2532 (ST124) . 

25 The images of the Fourier transformed partial areas 

are combined by the combining unit 2534 to generate the 



combined Fourier image data (ST125) , the amplitude 
component is eliminated by the phase extraction unit 2535, 
then the remaining phase component is extracted (ST126) , 
and inverse Fourier transform processing is carried out 
5 by the inverse Fourier transform unit 2536 (ST127) , 
whereby the correlation image data S253 is generated. 

At step ST128, the correlation value generation 
unit 241 generates a correlation value in accordance with 
the correlation image data S253. The judgment unit 242 

10 judges whether or not the correlation value S241 is 

larger t:han a predetermined threshold value (ST129) . When 
it is larger than the threshold value, it judges that the 
registered image 'AIM and the matching image RIM coincide 
(ST130) , and the operation processing unit 16 performs 

15 the predetermined processing. On the other hand, when the 
judgment unit 242 judges at step ST129 that the 
correlation value S241 is smaller than the threshold 
value (ST131) , it judges that the registered image AIM 
and the matching image RIM do not coincide and terminates 

20 the series of processing. 

As mentioned above, in the present embodiment, 
since the positional deviation between the registered 
image AIM and the matching image RIM is detected, the 
common pixel areas of each are extracted, and the 

25 correlation between the common pixel areas is found, in 
comparison with the case where the correlation is found 



between the pixel areas of the entire images in for 
example the first embodiment, a reduction of the 
correlation peak strength by the non-common portions of 
the two images can be avoided and the matching precision 
5 can be raised. 

Further, in the present embodiment, partial pixel 
areas set in advance were extracted when extracting the 
common areas, but the invention is not limited to this 
format. For example, it is also possible to select the 

10 areas so that the common areas of the two images become 
the maximum based on the amount of positional deviation 
between the registered image AIM and the matching image 
RIM and find the correlation strength of the extracted 
areas. By doing this, the correlation peak strength can 

15 be made higher and the precision of matching can be 
raised. 

FIG. 13 is a functional block diagram of an image 
matching r system according to a third embodiment of the 
present invention. An image matching system lb according 

20 to the present embodiment performs processing for 

correction of scaling and rotation of the registered 
imagie AIM and the matching image RIM, then, when there is 
an amount of deviation of parallel movement between the 
two images within a range set in advance, executes 

25 correlation value generation processing. It does not 

generate a correlation value when the amount of deviation 



of parallel movement is out of the range. The difference 
between the image matching system lb and the image 
matching system 1 according to the first embodiment 
resides in a matching unit 24b. The rest of the 
5 components are the same, so explanations will be omitted. 
The matching unit 24b has, as shown in FIG. 13, a 
parallel movement detection unit 2411, a parallel 
movement judgment unit 2412, a correlation value 
generation unit 241, and a judgment unit 242. 

10 FIG. 14 is a view for explaining the operation of 

detecting the amount of parallel movement of the parallel 
movement detection unit. The parallel movement detection 
unit 2411 detects the amount* of parallel movement S2411 
based on the signal S23 indicating the correlation image 

15 data output from the parallel movement unit 23. The 
parallel movement detection unit 2411 detects, for 
example, as shown in FIG. 14, the amount of deviation of 
the correlation peak position PP from the center position 
"cnt" based on the correlation image data S23. In FIG. 14, 

20 the peak position PP is indicated by a black square, and 
the center position "cnt" is indicated by a white square. 
The amount of deviation from the center position "cnt" is 
equivalent to the amount of parallel movement between the 
registered image AIM and the matching image RIM. 

25 The parallel movement judgment unit 2412 judges 

that the registered image AIM and the matching image RIM 



do not coincide when the amount of deviation (amount of 
parallel movement) S2411 is larger than a threshold value 
set in advance based on the amount of deviation (amount 
of parallel movement) S2411 detected by the parallel 
5 movement detection unit 2411 . In more detail, the 

parallel movement judgment unit 2412 judges whether or 
not the peak position PP is located in the window WD 
surrounding the center position "cnt" in advance in the 
correlation image data S23 as shown in FIG. 14. This 

10 window WD corresponds to the threshold value concerning 
the amount of parallel movement mentioned above. 

The correlation value generation unit 241 generates 
a correlation value S241 based on the correlation peak 
strength when it is judged by the parallel movement 

15 judgment unit 2412 that the correlation peak position PP 
exists in this window WD. The judgment unit 242 judges 
the matching of the images based on the correlation value 
S241 generated by the correlation value generation unit 
241 and outputs data S242 indicating the result of the 

20 judgment. For example, the judgment unit 242 compares it 
with a threshold value set in advance and judges that the 
registered image AIM and the matching image RIM coincide 
when the correlation value S241 is larger than the 
threshold value, while judges that they do not coincide 

25 when the correlation value S241 is smaller than the 
threshold value. 



FIG. 15 is a flow chart for explaining the 
operation of the image matching system shown in FIG, 14. 
The operation of the image matching system lb will be 
simply explained with reference to FIG. 15. 
5 The processing up to step ST19 is the same as the 

operation of the image matching system 1 according to the 
first embodiment, so the explanation thereof will be 
omitted. At step ST220, the amount of parallel movement 
S2411 is detected from the signal S23 indicating the 

10 correlation strength image generated at the inverse 

Fourier transform unit 234 at step ST19 by the parallel 
movement detection unit 2411. At step ST221, the parallel 
movement judgment unit 2412 judges whether or not the 
peak position PP is located within the window WD 

15 surrounding the center position "cnt" set in advance 
based on the amount of parallel movement S2411. 

When the parallel movement detection unit 2411 
judges that the peak position PP is located in the window 
WD, the correlation value generation unit 241 generates 

20 the value obtained by dividing the sum of the correlation 
values in the correlation area comprised by 5x5 pixels 
in the vicinity of for example the peak position by the 
sum of the correlation values of the entire area as the 
correlation value of the two images as mentioned above 

25 based on the correlation strength image S23 (ST222) . 

At step ST223, the judgment unit 242 judges whether 



or not the correlation value S241 generated by the 
correlation value generation unit 241 is larger than a 
predetermined threshold value. When it is larger than the 
threshold value, it judges that the registered image AIM 
5 and the matching image RIM coincide (ST224) . On the other 
hand, at step ST223, when the correlation value S241 is 
smaller than the threshold value, and at step ST221, when 
the parallel movement detection unit 2411 judges that the 
peak position PP is not located in the window WD, it 

10 judges that the two images do not coincide (ST225) and 
terminates the series of processings. 

As explained above, in the present embodiment, 
since provision was made of 4 the parallel movement 
detection unit 2411 for detecting the amount of parallel 

15 movement based on the correlation image data S23, the 

parallel movement judgment unit 2412 for judging whether 
or not the amount of parallel movement is within a 
threshold value based on the detected amount of parallel 
movement, the correlation value generation unit 241 for 

20 generating a correlation value based on the correlation 
peak strength when the amount of parallel movement is 
within the threshold value, and the judgment unit 242 for 
matching two images based on the correlation value, when 
there is no correlation between the images at all, the 

25 correlation peak PP of the correlation image data is 

rarely located near the image center. Accordingly, when 



the correlation peak position PP is out of the 
predetermined window WD based on the correlation image 
data, the following processings such as the correlation 
value generation processing are not carried out, so the 
5 processing load can be reduced. 

Further, since the image matching system lb 
performs matching when the correlation strength peak is 
located inside the predetermined area including the 
center position "cnt" in the correlation image data and 

10 the peak strength is larger than a predetermined 

threshold value, the matching precision can be made 
higher than that of the image matching system of the 
first embodiment. 

FIG. 16 is a view of part of the functional blocks 

15 of an image matching system according to a fourth 

embodiment the present invention . An image matching 
system lc according to the present embodiment has a 
function of performing processing when the amount of 
parallel movement is within a predetermined threshold 

20 value of the image matching system lb of the third 
embodiment and a function of extracting the common 
partial areas of the image matching system la of the 
second embodiment. 

A CPU 15c according to the present embodiment has a 

25 scalar information— rotation information unit 21, a 
correction unit 22, a parallel movement unit 23, a 



parallel movement detection unit 251, a parallel movement 
judgment unit 2512, a common partial area extraction unit 
252, a phase only correlation unit 253, and a matching 
unit 24 . The scalar information— rotation information unit 
5 21, the correction unit 22, and the parallel movement 
unit 23 have the same functions as those of the image 
matching system 1 according to the first embodiment, so 
are not illustrated. 

In FIG. 16, the parallel movement detection unit 

10 251 and the parallel movement judgment unit 2512 have the 
same functions as those of the parallel movement 
detection unit 2411 and the parallel movement judgment 
unit 2412 of the image matching system lb according to 
the third embodiment. 

15 The parallel movement detection unit 251 detects 

the amount of parallel movement based on the correlation 
peak position in the correlation strength image data S23 
output from the parallel movement unit 23. The parallel 
movement judgment unit 2512 judges whether or not the 

20 amount of parallel movement is within a predetermined 
threshold value, in more detail, whether or not the 
correlation peak position is located in the predetermined 
window WD based on the amount of parallel movement 
detected at the parallel movement unit 23, and judges 

25 that the two images do not coincide when the amount of 
parallel movement is larger than the predetermined 



threshold value. 

Further, in FIG. 16, the common partial area 
extraction unit 252 and the phase only correlation unit 
253 have substantially the same functions as those of the 
5 common partial area extraction unit and the phase only 
correlation unit of the image matching system la 
according to the second embodiment. The difference 
resides in that the common partial area extraction unit 
252 extracts the common partial pixel areas from the 

10 registered image AIM and the matching image RIM when it 
is judged at the parallel movement judgment unit 2512 
that the amount of parallel movement is within a 
predetermined threshold value. 

The matching unit 24 has, as shown in FIG. 16, a 

15 parallel movement detection unit 2411, a parallel 
movement judgment unit 2412, a correlation value 
generation unit 241, and a judgment unit 242. The 
matching unit 24 is substantially the same as the 
matching unit 24 of the image matching system 1c 

20 according to the third embodiment. The difference resides 
in the point that processing s carried out for matching 
for the common partial areas . 

FIG. 17 is a flow chart for explaining the 
operation of the image matching system 1c according to a 

25 fourth embodiment. Only the difference of the operation 
of the image matching system lc from the image matching 



system lb according to the third embodiment will be 
explained with reference to FIG. 17. 

The processings of steps ST1 to ST19 and up to 
ST220 are the same as in the operation of the image 
5 matching system lb according to the third embodiment 

shown in FIG. 15, so the explanations will be omitted. At 
step ST220, the amount of parallel movement S251 is 
detected from the signal S23 indicating the correlation 
strength image generated at the inverse Fourier transform 

10 unit 234 at step ST19 by the parallel movement detection 
unit 251. At step ST231, the parallel movement judgment 
unit 2512 judges whether or hot the peak position PP is 
located within the window WD surrounding the center 
position "cnt" set in advance based on the amount of 

15 parallel movement S251 (ST231) . 

When the amount of parallel movement judgment unit 
251 judges at step ST231 that the peak position P is 
located within the window WD, the partial extraction unit 
2521 extracts the partial area set in advance from the 

20 registered image AIM (ST232) . This partial area is 

Fourier transformed by the Fourier transform unit 2531 
(ST233) . The partial extraction unit 2522 extracts the 
common partial area corresponding bo the partial area of 
the registered image AIM from the matching image RIM 

25 based on the amount of parallel movement S251 (ST234) , 
and the Fourier transform unit 2532 performs Fourier 



transform processing (ST235) . 

The Fourier transformed images of the partial areas 
are combined by the combining unit 2534 to generate the 
combined Fourier image data (ST256) . The amplitude 
5 component is removed by the phase extraction unit 2535, 
then the remaining phase component is extracted (ST237) . 
The result is subjected to the inverse Fourier transform 
processing by the inverse Fourier transform unit 2536 to 
generate the correlation image data S253 (ST238) . 

10 At step ST239, the parallel movement detection unit 

2411 detects the amount of parallel movement S2411 of the 
common partial pixel areas . The parallel movement 
judgment unit 2412 judges whether or not the peak 
position PP is located within the window WD surrounding 

15 the center position "cnt" set in advance based on the 
amount of parallel movement' S24 11 (ST240) . When it is' 
judged that the peak position PP is located in the window 
WD, the correlation value generation unit 241 generates a 
value obtained by dividing the sum of the correlation 

20 values in the correlation area comprised by 5x5 pixels 
in the vicinity of for example the peak position by the 
sum of the correlation values of the common partial area 
as mentioned above based on the correlation strength 
image S25 (ST241) 

25 The judgment ' unit 242 judges whether or not the 

correlation value S241 is a predetermined threshold value 
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or more (ST242) . When it is larger than the threshold 
value, it judges that the registered image AIM and the 
matching image RIM coincide (ST243) , and the operation 
processing unit 16 performs the predetermined processing. 
On the other hand, when the judgment unit 242 judges at 
step ST242 that the correlation value S241 is smaller 
than the threshold value and when it is judged at step 
ST240 that the peak position PP is not located in the 
window WD surrounding the center position "cnt" set in 
advance (ST244) , the judgment unit 242 judges that the 
registered image AIM and the matching image RIM do not 
coincide and terminates the series of processing. 

As explained above, since the image matching system 
lc provides the function of performing processing when 
the amount of parallel movement is smaller than the 
threshold value of the image matching system lb according 
to the third embodiment and the function of extracting 
the common partial areas of the image matching system la 
of the second embodiment, it can perform higher precise 
matching. 

The image matching system lc according to the 
fourth embodiment provides a first limitation that the 
amount of parallel movement concerning all areas be not 
more than a predetermined threshold value and a second 
limitation that the amount of parallel movement 
concerning the common partial pixel areas be not more 



than a predetermined threshold value, therefore can 
perform matching having a higher precision. 

Further, it was provided with a parallel movement 
detection unit and a parallel movement judgment unit so 
5 as to perform the processing of the latter stages only in 
the case when the amount of parallel movement was smaller 
than the threshold value, therefore the processing load 
is reduced. Further, the processing time is shortened. 
Further, in comparison with the case of reliably 

10 calculating the correlation value every time, the rate of 
erroneous judgment due to the correlation value between 
images having no correlation exceeding the threshold 
value can be greatly reduced. 

Note that the present invention is not limited to 

15 the present embodiments. Any modifications are possible. 

In the first embodiment, the Fourier transform processing, 
the log transform processing, and the log-polar 
coordinate transform were carried out, the amount of 
parallel movement in the Fourier-Mellin space was 

20 calculated, and the scalar information and the rotation 
angle information were calculated, but the present 
invention is not limited to this format. For example, it 
is also possible to perform the coordinate transform in a 
space able to detect the scalar information and the 

25 rotation angle information. 

In the fourth embodiment, the predetermined 
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threshold value of the first limitation and the 
predetermined threshold value of the second limitation 
were the same, in more detail, they were based on the 
same window WD, but the present invention is not limited 
to this. For example, it is also possible to narrow down 
the candiates for calculation of the correlation value by 
making the threshold value concerning the first 
limitation according to all areas larger to loosen the 
limitation and making the threshold value concerning the 
second limitation according to the partial area smaller. 

Summarizing the effects of the invention, according 
to the present invention, ah image matching system and an 
image matching method and program able to perform 
matching even when there are differences in the amount of 
parallel movement, the rotation angle, and scaling 
between images can be provided. 

While the invention has been described with 
reference to specific embodiments chosen for purpose of 
illustration, it should be apparent that numerous 
modifications could be made thereto by those techniqueed 
in the art without departing from the basic concept and 
scope of the 1 invention . 



